Automatic screen brightness and camera exposure adjustment for remote multimedia collaboration sessions

ABSTRACT

The present disclosure provides methods and systems related to automatic adjustment of screen brightness for optimized presentation to both physically present and remote audience during the multimedia collaboration session. In one aspect, a method includes detecting presence of a screen in the field of view of a camera in a meeting room; determining if exposure of the camera or brightness of the screen is to be adjusted, to yield a determination; and controlling at least one of the exposure of the camera or the brightness of the screen based on the determination such that viewing of meeting room and the screen are legible for one or more audience and the screen is legible for one or more audience present in the meeting room.

TECHNICAL FIELD

The present technology pertains to managing remote multimediacollaboration sessions and more specifically to automatic adjustment ofscreen brightness and camera exposure for optimized presentation to bothphysically present and remote audience during the multimediacollaboration session.

BACKGROUND

In today's interconnected world, video conferencing presents a verysuitable option for many users located in different geographicallocations to communicate and collaborate. Step by step, advancements intechnologies related to video conferencing enable users to have anexperience that resembles in person meetings where all users arephysically present in a single location, can listen to otherparticipants, present material and collaborate.

Use of digital screens and boards such as smart whiteboards during suchcollaboration/video conferencing sessions is typical. As such digitalscreens are typically in the field of view of camera(s) that arebroadcasting the sessions to remote audience and the digital screens canappear very bright. Adjusting the brightness of the digital screens foraudience who are physically present in the room may result inoverexposure (bright and unreadable) of the digital screens for theremote audience through the broadcasting cameras while proper brightnessof the digital screens for the remote audience results in underexposure(dark and unreadable) of the digital screens for physically presentaudience. Furthermore, adjusting screen brightness or camera exposuremay also negatively impact, for the remote audience, depiction of the aconference room and physically present audience in the conference room.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features of the disclosure can be obtained, a moreparticular description of the principles briefly described above will berendered by reference to specific embodiments that are illustrated inthe appended drawings. Understanding that these drawings depict onlyexample embodiments of the disclosure and are not therefore to beconsidered to be limiting of its scope, the principles herein aredescribed and explained with additional specificity and detail throughthe use of the accompanying drawings in which:

FIG. 1 illustrates a multimedia collaboration setting, according to oneaspect of the present disclosure;

FIGS. 2A-B illustrate multimedia collaboration settings, according toone aspect of the present disclosure;

FIG. 3 provides a visual example of content over/under exposure problem;

FIG. 4 illustrates a method of automatically adjusting screen brightnessof one or more digital screens of the video conferencing session,according to one aspect of the present disclosure; and

FIG. 5 illustrates example components and system architectures,according to aspects of the present disclosure.

DETAILED DESCRIPTION

Various embodiments of the disclosure are discussed in detail below.While specific implementations are discussed, it should be understoodthat this is done for illustration purposes only. A person skilled inthe relevant art will recognize that other components and configurationsmay be used without parting from the spirit and scope of the disclosure.

References to one or an embodiment in the present disclosure can be, butnot necessarily are, references to the same embodiment; and, suchreferences mean at least one of the embodiments.

Reference to “one embodiment” or “an embodiment” means that a particularfeature, structure, or characteristic described in connection with theembodiment is included in at least one embodiment of the disclosure. Theappearances of the phrase “in one embodiment” in various places in thespecification are not necessarily all referring to the same embodiment,nor are separate or alternative embodiments mutually exclusive of otherembodiments. Moreover, various features are described which may beexhibited by some embodiments and not by others. Similarly, variousfeatures are described which may be features for some embodiments butnot other embodiments.

The terms used in this specification generally have their ordinarymeanings in the art, within the context of the disclosure, and in thespecific context where each term is used. Alternative language andsynonyms may be used for any one or more of the terms discussed herein,and no special significance should be placed upon whether or not a termis elaborated or discussed herein. Synonyms for certain terms areprovided. A recital of one or more synonyms does not exclude the use ofother synonyms. The use of examples anywhere in this specificationincluding examples of any terms discussed herein is illustrative only,and is not intended to further limit the scope and meaning of thedisclosure or of any exemplified term. Likewise, the disclosure is notlimited to various embodiments given in this specification.

Without intent to limit the scope of the disclosure, examples ofinstruments, apparatus, methods and their related results according tothe embodiments of the present disclosure are given below. Note thattitles or subtitles may be used in the examples for convenience of areader, which in no way should limit the scope of the disclosure. Unlessotherwise defined, technical and scientific terms used herein have themeaning as commonly understood by one of ordinary skill in the art towhich this disclosure pertains. In the case of conflict, the presentdocument, including definitions will control.

Although the terms first, second, etc. may be used herein to describevarious elements, these elements should not be limited by these terms.These terms are only used to distinguish one element from another. Forexample, a first element could be termed a second element, andsimilarly, a second element could be termed a first element, withoutdeparting from the scope of this disclosure. As used herein, the term“and/or,” includes any and all combinations of one or more of theassociated listed items.

When an element is referred to as being “connected,” or “coupled,” toanother element, it can be directly connected or coupled to the otherelement or intervening elements may be present. By contrast, when anelement is referred to as being “directly connected,” or “directlycoupled,” to another element, there are no intervening elements present.Other words used to describe the relationship between elements should beinterpreted in a like fashion (e.g., “between,” versus “directlybetween,” “adjacent,” versus “directly adjacent,” etc.).

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used herein, thesingular forms “a”, “an”, and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willbe further understood that the terms “comprises”, “comprising,”,“includes” and/or “including”, when used herein, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof.

It should also be noted that in some alternative implementations, thefunctions/acts noted may occur out of the order noted in the figures.For example, two figures shown in succession may in fact be executedsubstantially concurrently or may sometimes be executed in the reverseorder, depending upon the functionality/acts involved.

Specific details are provided in the following description to provide athorough understanding of embodiments. However, it will be understood byone of ordinary skill in the art that embodiments may be practicedwithout these specific details. For example, systems may be shown inblock diagrams so as not to obscure the embodiments in unnecessarydetail. In other instances, well-known processes, structures andtechniques may be shown without unnecessary detail in order to avoidobscuring example embodiments.

In the following description, illustrative embodiments will be describedwith reference to acts and symbolic representations of operations (e.g.,in the form of flow charts, flow diagrams, data flow diagrams, structurediagrams, block diagrams, etc.) that may be implemented as programservices or functional processes include routines, programs, objects,components, data structures, etc., that perform particular tasks orimplement particular abstract data types and may be implemented usinghardware at network elements. Non-limiting examples of such hardware mayinclude one or more Central Processing Units (CPUs), digital signalprocessors (DSPs), application-specific-integrated-circuits, fieldprogrammable gate arrays (FPGAs), computers or the like.

Additional features and advantages of the disclosure will be set forthin the description which follows, and in part will be obvious from thedescription, or can be learned by practice of the herein disclosedprinciples. The features and advantages of the disclosure can berealized and obtained by means of the instruments and combinationsparticularly pointed out in the appended claims. These and otherfeatures of the disclosure will become more fully apparent from thefollowing description and appended claims, or can be learned by thepractice of the principles set forth herein.

1. Overview

The present disclosure describes several example embodiments directed tooptimizing brightness of screens/exposures of cameras that are used toconvey information in a multimedia collaboration session, for both inperson users present in a physical location in which the screens areinstalled as well as remote users who are connected to and are able toview the screens using their remotely connected multimedia devices.

In one aspect, a method includes detecting presence of a screen in thefield of view of a camera in a meeting room; determining if exposure ofthe camera or brightness of the screen is to be adjusted, to yield adetermination; and controlling at least one of the exposure of thecamera or the brightness of the screen based on the determination suchthat viewing of meeting room and the screen are legible for one or moreaudience and the screen is legible for one or more audience present inthe meeting room.

In another aspect, a system includes a screen; a camera configured tocapture live images within a room for transmission to one or more remoteend users; and a controller. The controller is configured to detectpresence of the screen; determine if exposure of the camera orbrightness of the screen is to be adjusted, to yield a determination;and control at least one of the exposure of the camera or the brightnessof the screen based on the determination such that viewing of meetingroom and the screen are legible for one or more audience and the screenis legible for one or more audience present in the meeting room.

In another aspect, a non-transitory computer-readable medium hascomputer-readable instructions, which when executed by at least oneprocessor, causes the at least one processor to detect presence of ascreen in a field of view of a camera in a meeting room; determine ifexposure of the camera or brightness of the screen is to be adjusted, toyield a determination; and control the exposure of the at least onecamera or the brightness of the screen based on the determination suchthat viewing of meeting room and the screen are legible for one or moreaudience and the screen is legible for one or more audience present inthe meeting room.

2. Description

The present disclosure provides methods and systems related to automaticadjustment of screen brightness and/or camera exposure for optimizedpresentation to both physically present and remote audience during themultimedia collaboration session. The disclosure begins with adescription of example settings in which the concepts presented hereinmay be implemented.

FIG. 1 illustrates a multimedia collaboration setting, according to oneaspect of the present disclosure. In an example of a multimediacollaboration session using video conferencing, setting 100 includesthree separate parties participating in the multimedia collaborationsession. Setting 100 includes a conference room 102, a remote mobiledevice 104 and another conference room 106. While FIG. 1 illustrates anexample three separate parties, the present disclosure is not limitedthereto and there may be more or less parties participating in themultimedia collaboration session.

The conference rooms 102 and 106 and the mobile device 104 are remotelyconnected to one another through the appropriate local area connectionsand over the internet, as is known (e.g., cloud based collaborationserver 128). In other words, the conference rooms 102 and 106 and themobile device 104 are located in different geographical locations.

Conference room 102 includes a table 108, with one or more audience 110sitting at table 108. There may also be at least one other audience,which may be referred to as presenter 112 who may have material contentto present to audience 110 and/or any other remotely connected audiencesuch as audience present in conference room 106 and/or audienceassociated with mobile device 104. Presence of presenter 112 is optionaland the concepts disclosed herein are applicable to situations where noparticular presenter is making a presentation to audience 110 and/orremote audience.

Conference room 102 may further include display 114 and main camera 116.Display 114 may be a digital display such as a digital whiteboard and/orany other display device that may be used by presenter 112 to presentcontent/make presentations to audience 110 and/or remote audience.Display (screen) 114 may be any known or to be developed display devicecapable of presenting content to audience. Display 114 may have anadjustable brightness that may be controlled by processing unit(controller) 124, as will be further described below with reference toFIG. 4.

Optionally, display 114 can be configured to present a view of otherremote participating parties (e.g., audience using mobile device 104and/or audience in conference room 106). In such case, display 114 mayhave a display section with a plurality of thumbnail display sections,each of which may display a view of a current speaker during themultimedia collaboration session. For example, when a participantassociated with mobile device 104 speaks, display 114 can display a viewof the participating associated with mobile device 104 (which may alsoinclude the surrounding areas of the participant visible through acamera of mobile device 104). At the same time, each of thumbnaildisplay sections represents a small version of a view of each differentremote location and its associated participants taking part in the videoconferencing session. For example, assuming that conference room 102 isa branch of company A located in New York and conference room 106 isanother branch of company A located in Los Angeles and mobile device 104is associated with an employee of company A teleworking from Seattle,then one of thumbnail display regions corresponds to a view ofconference room 102 and its participants as observed by main camera 116,another one of thumbnail display regions corresponds to a view ofconference room 106 and its participants as observed by camerasinstalled therein and another one of thumbnail display regionscorresponds to a view of the teleworking employee of company A usingmobile device 104. Furthermore, each thumbnail display region can have asmall caption identifying a geographical location of each of conferencerooms 102 and 106 and mobile device 104 (e.g., New York office, LosAngeles office, Seattle, Wash., etc.).

Main camera 116 may operate to provide a view of conference room 102 toremote audience at mobile device 104 and/or in conference room 106. Inone example, main camera 116 may utilize a technology capable oftracking and focusing on presenter(s) and/or individual audience 110 whomay be actively participating, commenting, speaking during themultimedia collaboration session, etc. This tracking system can be theSpeakerTrack system developed, manufactured and distributed by CiscoTechnology, Inc., of San Jose, Calif. For example, if one audience 110is currently speaking, main camera 116 can zoom in (and/or tilthorizontally, vertically, diagonally, etc.) in order to present/capturea focused stream of audience 110 to participants at mobile device 104and/or conference room 106, a close up version of audience 110 ratherthan a view of the entire conference room 102 which results in audience110 and/or other users/presenters being shown relatively smaller (whichmakes it more difficult for remote participants to determine accuratelywho the current speaker in conference room 102 is). In one example,instead of one, there may be a pair of main cameras 116 utilized forbetter tracking of users/speakers inside conference room 102.

In one example, conference room 102, mobile device 104 and conferenceroom 106 may be connected to cloud based collaboration server 128.Collaboration server 128 can have one or more processors and one or morememories to implement functionalities thereof to ensure establishing andmaintaining connections between conference rooms 102 and 106 and mobiledevice 104 during the multimedia collaboration session.

Furthermore, collaboration server 128 can be used by a participant tosetup the multimedia collaboration session (e.g., create an event with agiven date and duration as well as identification of participants,etc.). The created event may then be shared with identified participantsor associated parties via for example, an electronic mail, a textmessage, an application-based notification, etc. Identified participantscan then interact with the invitation to accept, reject or provide anyother indication with respect to their participation status in theevent.

While certain components and number of different elements are describedas being included in setting 100, the present disclosure is not limitedthereto. For example, there may be more or less participantsparticipating in a video conferencing session via their correspondingdevices than that shown in FIG. 1. There may be more or lessparticipants in each conference room shown in FIG. 1. Mobile device 104is not limited to being a mobile telephone device but can instead be anyknown or to be developed mobile computing device having necessarycomponents (e.g., a microphone, a camera, a processor, a memory, awired/wireless communication means, etc.) for communicating with otherremote devices in a video conferencing session.

Furthermore, software/hardware for enabling the video conferencingsession may be provided by various vendors such as Cisco Technology,Inc. of San Jose, Calif. Such software program may have to be downloadedon each device or in each conference room prior to being able toparticipate in an online video conferencing session. By installing suchsoftware program, participants can create, schedule, log into, recordand complete one or more video conferencing sessions.

FIG. 1 describes above provides an example setting with a camera havingthe entire conference room 102 including its users, presenter, screenand main camera in its FOV. Furthermore, example setting of FIG. 1 isone where cameras and display are coupled to the same audio/video codecfor adjusting exposure of camera 122 and/or brightness of display 114and ultimately optimizing view of display 114 as well as conference room102 and audience 110 for physically present as well as remote users.

Alternatively, setting 100 of FIG. 1 may be replaced with one, where notall components are connected to the same audio/video codec but insteadseveral disparate systems (digital whiteboards, cameras, displays, etc.)may be utilized in a conference room and more broadly in a multimediacollaboration session. Examples of such settings to which the presentdisclosure is applicable will be described below with reference to FIGS.2A-B.

FIGS. 2A-B illustrate multimedia collaboration settings, according toone aspect of the present disclosure. In describing FIGS. 2A-B,components that are the same as those described above with reference toFIG. 1, are numbered the same and thus will be not described further,for sake of brevity.

In setting 200 of FIG. 2A, components 104, 106, 108, 110, 112, 114, 116,118, 120 and 126 are the same as those described above with reference toFIG. 1 and thus will not be described again.

In setting 200 of FIG. 2A, in addition to display 114, there is anadditional digital board 204 installed on a sidewall of conference room202. Such digital board 204, similar to display 114, may have anadjustable brightness. While in FIGS. 2A-B, a digital board 204 isreferenced, digital board 204 may be the same as screen 114, which cansimply be a display showing content. For purposes of this disclosure,screen 114, digital board/whiteboard 204 may be used interchangeably andthe inventive concepts are equally applicable to both. Furthermore, insetting 200 and in contrast to setting 100 of FIG. 1, main camera 116may perform the functionalities of camera 122 of FIG. 1 and digitalboard 204 as well as audience 110, presenter 112, table 108, microphones118 and control unit 120 fall within FOV of main camera 116.

Furthermore and in contrast to FIG. 1, digital board 204, display 114,camera 116 as well as mobile device 104 and components in conferenceroom are each registered with cloud service (controller) 206. Cloudservice 206 may be, for example, WebEx service, developed andcommercially available via Cisco Technology, Inc. of San Jose, Calif.With Cisco WebEx service as a non-limiting example, various componentsmay be communicatively coupled via WebEx Lyra service.

In setting 250 of FIG. 2B, components 104, 106, 108, 110, 112, 114, 116,118, 120, 126, 204 and 206 are the same as those described above withreference to FIG. 2A and thus will not be described again for sake ofbrevity.

The only difference between setting 200 of FIG. 2A and setting 250 ofFIG. 2B is that digital board 204 is installed on the back wall ofconference room 252 directly opposite of (and in FOV of) main camera116, whereas in FIG. 2A, digital board 204 is installed on a side wallof conference room 202.

Furthermore and in contrast to FIG. 1, digital board 204, display 114,camera 116 as well as mobile device 104 and components in conferenceroom are each registered with cloud service 206. Cloud service 206 maybe, for example, WebEx service, developed and commercially available viaCisco Technology, Inc. of San Jose, Calif. With Cisco WebEx service as anon-limiting example, various components may be communicatively coupledvia WebEx Lyra service.

It should be noted that while FIGS. 1 and 2A-B provide a single display114/digital board 204 on which content may be displayed and brightnessthereof may be adjusted and a single camera 122/116 the exposure ofwhich may be adjusted in conjunction with brightness of thedisplay/digital board, the present disclosure is not limited thereto. Inother words, there may be multiple displays and cameras in a conferenceroom whose respective brightness and exposure may be automaticallyadjusted to optimize display of content for both in person and remoteaudience.

As briefly mentioned above, use of screens such as display 114 and/ordigital board 204 during multimedia collaboration/video conferencingsessions is prevalent. As such digital screens are typically in thefield of view of camera(s) that are broadcasting the sessions to remoteaudience the digital screens may appear very bright and unreadable.Adjusting the brightness of the digital screens for audience who arephysically present in the room may result in overexposure (bright andunreadable) of the digital screens for the remote audience through thebroadcasting cameras while proper exposure for the remote audienceresults in underexposure (dark and unreadable) for physically presentaudience. FIG. 3 provides a visual example of content over/underexposure problem.

FIG. 3 provides two snapshots of display 114 of FIG. 1, which wouldequally be applicable to digital board 204 of FIGS. 2A-B. Snapshot 300is display 114 as viewed by audience 110 present in conference room 102while snapshot 302 is display 114 as viewed by remote audience inconference room 106 and/or via mobile device 104. In this scenario,brightness of display 114 is adjusted to provide the rightexposure/visibility of display 114 to audience 110 (e.g., content (1, 2,3) provided on display 114 is legible and clearly visible to audience110). At the same time, auto exposure algorithms being implemented bycamera 122 (or similarly main camera 116 in FIGS. 2A-B) attempt toensure that faces of audience 110 and/or presenter 112 are properlycaptured for display to remote audiences at mobile device 104 and/or inconference room 106. Due to this attempt, content (e.g., 1, 2, 3) ondisplay 114 and/or digital board 204 are completely overexposed (brightand unreadable/blurry). As will be described below with reference toFIG. 4, it is possible to adjust exposure of camera 122 and/orbrightness of display 114 or digital whiteboard 204 to improvevisibility of content displayed on display 114 or digital whiteboard 204for the remote audience while maintaining legibility and visibilitythereof for audience 110 present in conference room 102.

FIG. 4 illustrates a method of automatically adjusting screen brightnessof one or more digital screens of the video conferencing session,according to one aspect of the present disclosure. FIG. 4 will bedescribed from perspective of processing unit 124 and more generallywith respect to FIG. 1. However, functionalities of processing unit 124may be implemented in a similar manner by cloud service 206 describedabove with reference to FIGS. 2A-B and FIG. 4 is equally applicable tosettings 200 and 252 of FIGS. 2A and B.

At S400, processing unit 124 detects presence of display 114 in FOV ofcamera 122 in conference room 102. Alternatively and with reference toexample settings of FIGS. 2A-B, cloud service 206 may detect thepresence of digital board 204 in FOV of main camera 116 in conferenceroom 202 or 252.

In one example, locating/detecting display 114 in the conference room102 may be performed via an installation wizard. For integratedendpoints including microphones and speakers, the ‘double active radar’method disclosed in U.S. application Ser. No. 15/729,160 filed on Oct.10, 2017 can be used, the entire content of which is incorporated hereinby reference. Another example detection method may be based on machinelearning, where over time location of screens and displays in conferenceroom 102 is learned (e.g., using sensors 126). Another example method,which can be combined with the machine learning method is for display114 to display known patterns on all available screens inside conferenceroom 102 and use known or to be developed computer vision techniques todetect the location of screens inside conference room 102.

Upon detection of display 114, at S402, processing unit 124 analyzescharacteristics of display 114 and camera 122.

These characteristics, at any given instance, are communicated to andthus are known to processing unit 124. More specifically, detectedcharacteristics can include capabilities of the display 114 and camera122, such as the range of screen brightness level of display 114, lensiris of camera 122, digital gain range of camera 122, etc., are known toprocessing unit 124.

Based on these characteristics of display 114 and camera 122, processingunit 124, at S404, determines target brightness (e.g., LCD backlightintensity) of display 114 (first target) and target exposure level andappropriate exposure settings for camera 122 (second target).

The above described example targets may be determined based onexperiments and/or empirical. For example, for target exposure level ofcamera 122, artificial intelligence (AI) may be utilized to determineappropriate exposure or pixel average for different models of camerasand different manufactures.

At S405, processing unit 124 analyzes a captured image of conferenceroom 102 to determine a current exposure level of camera 122 and/or acurrent brightness of display 114, according to any known or to bedeveloped image/video processing methods. For example, determining acurrent exposure level of camera 122 and/or current brightness ofdisplay 114 can be done by examining pixel values (e.g., a room may beexposed such that the average linear pixel value is 18%). By knowinglocation of display 114 in a captured image, processing unit 124 canexamine legibility by evaluating pixel values of the location of display114 within the captured image.

At S406, processing unit 124 determines if current brightness of display114 and/or current exposure of camera 122 (determined at S405) is/are tobe adjusted based on comparison of the current brightness of display 114and/or the current exposure of camera 122 to the corresponding one oftarget brightness and target exposure level. For example, if the acurrent exposure level of camera 122 is lower or higher than the abovedescribed target exposure level, then processing unit 124 may determineto adjust the exposure of camera 122.

The same analysis applies to display 114. For example, if the currentbrightness is higher or lower than the target brightness describedabove, processing unit 124 may determine to adjust the brightness ofdisplay 114.

In another example, the above described targets may constitute ranges(e.g., range of acceptable iris opening for camera 122, range ofacceptable LCD backlight intensity for display 114). While exposure ofcamera 122 and brightness of display 114 may be within a permissiblerange, processing unit 124 may still make adjustments to the exposure ofcamera 122 and/or brightness of display 114 in order to optimize/improvevisibility of content on display 114 for both in person/present as wellas remote audience and also ensure optimized and legible depiction ofconference room 102 (including audience 110) for the remote audience.For example, processing unit 124 may utilize a machine learning basedmodel, where over time and based on various data analysis (based on datacollected using sensors 126, for example) as well as user feedback,optimal exposure or camera 122 and brightness of display 114 may belearned and applied.

In another example, any changes to content of display 114, lighting ofconference room 102, movement of camera 122 and/or main camera 116 maytrigger re-evaluation and adjustment of exposure of camera 122 and/orbrightness of display 114.

If at S406, processing unit 124 determines that adjustments to exposureof camera 122 and/or brightness of display 114 are needed, then at S408,processing unit 124 controls exposure of camera 122 and/or brightness ofdisplay 114 such that legibility of display 114 for both physicallypresent and remote audience is optimized while at the same timedepiction of other elements (e.g., conference room 102 in general and inparticular audience 110) are also optimized for the remote audience.Achieving the correct exposure level in camera 122 may be done bymodifying analog gain, digital gain, integration time or moving iris ofcamera 122. Achieving correct brightness of display 114 may be done byreducing or increasing brightness level of display 114.

As a non-limiting example, a simple exposure algorithm may be designedto keep the pixel average for an entire scene of conference room 102 atmiddle gray intensity (e.g., 18% on linear pixel data). A differentnon-limiting approach is to look at parts of the scene (e.g. faces ofaudience) and keep the pixel average of these parts of the scene atmiddle gray intensity. In another example and for other parts in a scene(e.g., display 114), the algorithms may allow lower pixel averages fordisplay 114 relative to other parts such as faces. A few non-limitingexamples are provided below.

As an example, if current exposure level of camera 122 indicates thatconference room 102 and its content including audience 110 are correctlyexposed but display 114 is not (e.g., too bright), then processing unit124 may only adjust (e.g., lower) brightness of display 114 in anattempt to improve legibility of display 114 for remote audience.

In another example, if current exposure level of camera 122 indicatesthat conference room 102 and its content including audience 110 areunderexposed but display 114 is correctly exposed, then processing unit124 may increase exposure of camera 122. However, increasing exposure ofcamera 122 alone may result in display 114 becoming less legible toremote audience. Therefore, in one example, simultaneous with increasingexposure of camera 122, processing unit 124 may also lower brightness ofdisplay 114 to adjust (compensate) for increased exposure of camera 122.

In another example, if current exposure level of camera 122 indicatesthat conference room 102 and its content including audience 110 areoverexposed but display 114 is correctly exposed, then processing unit124 may reduce exposure of camera 122. Reducing exposure of camera 122may allow for increasing brightness of display 114.

In another example, if current exposure level of camera 122 indicatesthat conference room 102 and its content including audience 110 areoverexposed and display 114 is not legible (e.g., overexposed orunderexposed), then processing unit 124 may reduce exposure of camera122 (e.g., gradually) and each time evaluate pixel values correspondingto display 114 to determine if display 114 is legible.

As another non-limiting example, if at S406, processing unit 124determines that a current exposure level of camera 122 is lower than theabove described target exposure level, then at S408, processing unit 124may increase camera 122's exposure level (e.g., increase Iris opening ofcamera 122 with one step, e.g. from f/2.8-f/2.0), in an attempt toadjust exposure of faces of audience in conference room 102 and improvebrightness and legibility of display 114 as well as conference room 102including audience 110 for remote audience while maintaining a legibledisplay of content on display 114 for audience present in conferenceroom 102. In another example, if at S406, processing unit 124 determinesthat a current exposure level of camera 122 is more than the targetexposure level, then at S408, processing unit 124 may lower camera 122'sexposure level to avoid bright and illegible display of content ondisplay 114 to the remote audience.

With respect to display 114, if at S406, processing unit 124 determinesthat the current brightness of display 114 is less than the targetbrightness described above, then at S408, processing unit 124 mayincrease brightness of display 114 (e.g., from 40% to 50%) to improveand adjust legibility of display 114 for both present and remoteaudience. Similarly, if at S406, processing unit 124 determines that thecurrent brightness of display 114 is more than the target brightnessdescribed above, then at S408, processing unit 124 may decrease thebrightness of display 114 (e.g., from 60% to 55%) to improve and adjustlegibility of display 114 for both present and remote audience. Thebrightness can be reduced in small steps while an auto exposurealgorithm evaluates the result of change in screen brightness. In suchcase and after each small step, the process reverts back to S402 andprocessing unit 124 performs S402, S404, S406 and S408 to evaluate andimplement the next small step. In another example, processing unit 124may estimate the required brightness reduction and reduce this in largersteps, with subsequent evaluation using the auto exposure algorithm. Animplementation of this could be to control the screen brightness via theauto exposure algorithm in camera 122. Accordingly, adjustments tobrightness of display 114 and exposure of camera 122 may constitute atrade-off between the values of two until processing unit 124 determinesno further adjustments to exposure level of camera 122 and/or brightnessof display 114 is/are needed.

In one example, controlling the brightness of display 114 (and/or board204) is possible using a proprietary API for integrated displays/systemsand HDMI Consumer Electronic Control (CEC) for non-integrated/generaldisplays/systems.

Components of setting 100 (e.g., display 114, main camera 116, camera122, etc.) form an integrated system where communication standards andextensions are defined by the same provider. For example, setting 100forms an integrated system when display 114, main camera 116, camera 122are developed and provided by Cisco Technologies, Inc. of San Jose,Calif. Accordingly, the proprietary API may be XAPI developed by CiscoTechnologies, Inc. of San Jose, Calif.

In another example, components of setting 100 (e.g., display 114, maincamera 116, camera 122, etc.) form a non-integrated/general system whenone or more components such as display 114, main camera 116, camera 122,etc. are provided by different providers (e.g., display 114 is not aCisco provided display). For such non-integrated displays, controllingthe brightness of display 114 may be performed via HDMI CEC with adefined extension supported by manufacturer/provider of such display114.

Once adjustments are completed, process of FIG. 4 returns to S400 (oroptional to S402) and processing unit 124 may periodically or upondetection of a triggering condition, repeat S400, S402, S404, S406 andS408 (or alternatively S402, S404, S406 and S408, if the process revertsback to S402). Such triggering condition can include, but is not limitedto, changes to content of display 114, lighting of conference room 102,movement of camera 122 and/or main camera 116, etc.

Referring back to S406, if at S406, processing unit 124 determines thatadjustments to exposure of camera 122 and/or brightness of display 114are not needed, then process of FIG. 4 returns to S400 (or optionally toS402) and processing unit 124 implements S400, S402, S404, S405, S406and S408 (or alternatively S402, S404, S406 and S408, if the processreverts back to S402), as described above.

With above examples of automatic adjustments to camera exposures anddisplay brightnesses for optimizing visibility of content on display forboth present and remote audience, the disclosure now turns todescription of example components and system architecture for implementelements in settings 100, 200 and 250 of FIGS. 1 and 2A-B including, butnot limited to, processing unit 124, cloud service 206, etc.

FIG. 5 illustrates example components and system architectures,according to aspects of the present disclosure. FIG. 5 illustratescomputing system architecture for use in setting of FIG. 1 and/or FIGS.2A-B, according to an aspect of the present disclosure. Computing systemarchitecture (device) 500 has components that are in electricalcommunication with each other using a system connection 505, such as abus. Exemplary computing device 500 includes a processing unit (CPU orprocessor) 510 and a system connection 505 that couples various systemcomponents including the system memory 515, such as read only memory(ROM) 520 and random access memory (RAM) 525, to the processor 510. Thecomputing device 500 can include a cache of high-speed memory connecteddirectly with, in close proximity to, or integrated as part of theprocessor 510. The computing device 500 can copy data from the systemmemory 515 and/or the storage device 530 to the cache 512 for quickaccess by the processor 510. In this way, the cache can provide aperformance boost that avoids processor 510 delays while waiting fordata. These and other modules can control or be configured to controlthe processor 510 to perform various actions. Other system memory 515may be available for use as well. The system memory 515 can includemultiple different types of memory with different performancecharacteristics. The processor 510 can include any general purposeprocessor and a hardware or software service, such as service (SVC) 1532, service (SVC) 2 534, and service (SVC) 3 536 stored in storagedevice 530, configured to control the processor 510 as well as aspecial-purpose processor where software instructions are incorporatedinto the actual processor design. The processor 510 may be a completelyself-contained computing system, containing multiple cores orprocessors, a bus, memory controller, cache, etc. A multi-core processormay be symmetric or asymmetric.

To enable user interaction with the computing device 500, an inputdevice 545 can represent any number of input mechanisms, such as amicrophone for speech, a touch-sensitive screen for gesture or graphicalinput, keyboard, mouse, motion input, speech and so forth. An outputdevice 535 can also be one or more of a number of output mechanismsknown to those of skill in the art. In some instances, multimodalsystems can enable a user to provide multiple types of input tocommunicate with the computing device 500. The communications interface540 can generally govern and manage the user input and system output.There is no restriction on operating on any particular hardwarearrangement and therefore the basic features here may easily besubstituted for improved hardware or firmware arrangements as they aredeveloped.

Storage device 530 is a non-volatile memory and can be a hard disk orother types of computer readable media which can store data that areaccessible by a computer, such as magnetic cassettes, flash memorycards, solid state memory devices, digital versatile disks, cartridges,random access memories (RAMs) 525, read only memory (ROM) 520, andhybrids thereof.

The storage device 530 can include services 532, 534, 536 forcontrolling the processor 510. Other hardware or software modules arecontemplated. The storage device 530 can be connected to the systemconnection 505. In one aspect, a hardware module that performs aparticular function can include the software component stored in acomputer-readable medium in connection with the necessary hardwarecomponents, such as the processor 510, system connection 505, outputdevice 535, and so forth, to carry out the function.

Regardless of the network device's configuration, it may employ one ormore memories or memory modules (including memory 506) configured tostore program instructions for the general-purpose network operationsand mechanisms for roaming, route optimization and routing functionsdescribed herein. The program instructions may control the operation ofan operating system and/or one or more applications, for example. Thememory or memories may also be configured to store tables such asmobility binding, registration, and association tables, etc. Memory 506could also hold various software containers and virtualized executionenvironments and data.

The network device 500 can also include an application-specificintegrated circuit (ASIC), which can be configured to perform routingand/or switching operations. The ASIC can communicate with othercomponents in the network device 500 via the bus 505, to exchange dataand signals and coordinate various types of operations by the networkdevice 500, such as routing, switching, and/or data storage operations,for example.

For clarity of explanation, in some instances the present technology maybe presented as including individual functional blocks includingfunctional blocks comprising devices, device components, steps orroutines in a method embodied in software, or combinations of hardwareand software.

In some embodiments the computer-readable storage devices, mediums, andmemories can include a cable or wireless signal containing a bit streamand the like. However, when mentioned, non-transitory computer-readablestorage media expressly exclude media such as energy, carrier signals,electromagnetic waves, and signals per se.

Methods according to the above-described examples can be implementedusing computer-executable instructions that are stored or otherwiseavailable from computer readable media. Such instructions can comprise,for example, instructions and data which cause or otherwise configure ageneral purpose computer, special purpose computer, or special purposeprocessing device to perform a certain function or group of functions.Portions of computer resources used can be accessible over a network.The computer executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, firmware, orsource code. Examples of computer-readable media that may be used tostore instructions, information used, and/or information created duringmethods according to described examples include magnetic or opticaldisks, flash memory, USB devices provided with non-volatile memory,networked storage devices, and so on.

Devices implementing methods according to these disclosures can comprisehardware, firmware and/or software, and can take any of a variety ofform factors. Typical examples of such form factors include laptops,smart phones, small form factor personal computers, personal digitalassistants, rackmount devices, standalone devices, and so on.Functionality described herein also can be embodied in peripherals oradd-in cards. Such functionality can also be implemented on a circuitboard among different chips or different processes executing in a singledevice, by way of further example.

The instructions, media for conveying such instructions, computingresources for executing them, and other structures for supporting suchcomputing resources are means for providing the functions described inthese disclosures.

Although a variety of examples and other information is used to explainaspects within the scope of the appended claims, no limitation of theclaims should be implied based on particular features or arrangements insuch examples, as one of ordinary skill would be able to use theseexamples to derive a wide variety of implementations. Further andalthough some subject matter may have been described in languagespecific to examples of structural features and/or method steps, it isto be understood that the subject matter defined in the appended claimsis not necessarily limited to these described features or acts. Forexample, such functionality can be distributed differently or performedin components other than those identified herein. Rather, the describedfeatures and steps are disclosed as examples of components of systemsand methods within the scope of the appended claims.

For clarity of explanation, in some instances the present technology maybe presented as including individual functional blocks includingfunctional blocks comprising devices, device components, steps orroutines in a method embodied in software, or combinations of hardwareand software.

Claim language reciting “at least one of” refers to at least one of aset and indicates that one member of the set or multiple members of theset satisfy the claim. For example, claim language reciting “at leastone of A and B” means A, B, or A and B.

We claim:
 1. A method comprising: determining that a screen is within a field of view (FOV) of a camera; determining a first exposure level associated with the screen and a second exposure level associated with a portion of a scene that is within the FOV of the camera, wherein the first exposure level and the second exposure level correspond to a current brightness of the screen and a current exposure of the camera; and adjusting at least one of the current exposure of the camera or the current brightness of the screen according to one or more first target exposure levels associated with the screen and one or more second target exposure levels associated with the portion of the scene.
 2. The method of claim 1, further comprising: determining the current brightness of the screen and the current exposure of the camera; comparing the first exposure level with the one or more first target exposure levels and the second exposure level with the one or more second target exposure levels; and determining a difference between at least one of (1) the first exposure level and the one or more first target exposure levels or (2) the second exposure level and the one or more second target exposure levels.
 3. The method of claim 2, wherein adjusting at least one of the current exposure of the camera or the current brightness of the screen is in response to determining the difference between at least one of the first exposure level and the one or more first target exposure levels or the second exposure level and the one or more second target exposure levels.
 4. The method of claim 1, wherein adjusting at least one of the current exposure of the camera or the current brightness of the screen comprises adjusting the current brightness of the screen to a target brightness associated with the one or more first target exposure levels associated with the screen.
 5. The method of claim 1, wherein adjusting at least one of the current exposure of the camera or the current brightness of the screen comprises adjusting the current exposure of the camera to a target exposure associated with the one or more second target exposure levels associated with the portion of the scene.
 6. The method of claim 1, wherein the one or more first target exposure levels comprises one of a first target exposure threshold or a first target exposure range, and wherein the one or more second target exposure levels comprises one of a second target exposure threshold or a second target exposure range.
 7. The method of claim 1, further comprising: analyzing an image of the scene captured by the camera; and determining the first exposure level associated with the screen and the second exposure level associated with the portion of the scene based on the image of the scene captured by the camera.
 8. A system comprising: one or more processors; and at least one non-transitory computer-readable medium having stored thereon instructions which, when executed by the one or more processors, cause the one or more processors to: determine that a screen is within a field of view (FOV) of a camera; determine a first exposure level associated with the screen and a second exposure level associated with a portion of a scene that is within the FOV of the camera, wherein the first exposure level and the second exposure level correspond to a current brightness of the screen and a current exposure of the camera; and control adjust at least one of the current exposure of the camera or the current brightness of the screen according to one or more first target exposure levels associated with the screen and one or more second target exposure levels associated with the portion of the scene.
 9. The system of claim 8, wherein the screen is a digital whiteboard, and wherein the system further comprises the camera.
 10. The system of claim 8, the at least one non-transitory computer-readable medium having stored thereon instructions which, when executed by the one or more processors, cause the one or more processors to: determine the current brightness of the screen and the current exposure of the camera; compare first exposure level with the one or more first target exposure levels and the second exposure level with the one or more second target exposure levels; and determine a difference between at least one of (1) the first exposure level and the one or more first target exposure levels or (2) the second exposure level and the one or more second target exposure levels.
 11. The system of claim 10, wherein adjusting at least one of the current exposure of the camera or the current brightness of the screen is in response to determining the difference between at least one of the first exposure level and the one or more first target exposure levels or the second exposure level and the one or more second target exposure levels.
 12. The system of claim 8, wherein adjusting at least one of the current exposure of the camera or the current brightness of the screen comprises adjusting the current brightness of the screen to a target brightness associated with the one or more first target exposure levels associated with the screen.
 13. The system of claim 8, the at least one non-transitory computer-readable medium having stored thereon instructions which, when executed by the one or more processors, cause the one or more processors to: continuously monitoring monitor the scene to detect changes in settings; and in response to a detected change, adjust at least one of the current exposure of the camera or the current brightness of the screen.
 14. The system of claim 8, the at least one non-transitory computer-readable medium having stored thereon instructions which, when executed by the one or more processors, cause the one or more processors to: analyze an image of the scene captured by the camera; and determine the first exposure level associated with the screen and the second exposure level associated with the portion of the scene based on the image of the scene captured by the camera.
 15. One or more non-transitory computer-readable media comprising computer-readable instructions, which when executed by one or more processors, cause the one or more processors to: detect that a screen is within a field of view (FOV) of a camera in; determine a first exposure level associated with the screen and a second exposure level associated with a portion of a scene that is within the FOV of the camera, wherein the first exposure level and the second exposure level correspond to a current brightness of the screen and a current; and adjust at least one of the current exposure of the camera or the current brightness of the screen according to one or more first target exposure levels associated with the screen and one or more second target exposure levels associated with the portion of the scene.
 16. The one or more non-transitory computer-readable media of claim 15, wherein the execution of the computer-readable instructions cause the one or more processors to: determine the current brightness of the screen and the current exposure of the camera; compare the first exposure level with the one or more first target exposure levels and the second exposure level with the one or more second target exposure levels; and determine a difference between at least one of (1) the first exposure level and the one or more first target exposure levels or (2) the second exposure level and the one or more second target exposure levels.
 17. The one or more non-transitory computer-readable media of claim 16, wherein adjusting at least one of the current exposure of the camera or the current brightness of the screen is in response to determining the difference between at least one of the first exposure level and the one or more first target exposure levels or the second exposure level and the one or more second target exposure levels.
 18. The one or more non-transitory computer-readable media of claim 15, wherein adjusting at least one of the current exposure of the camera or the current brightness of the screen comprises adjusting the current brightness of the screen to a target brightness associated with the one or more first target exposure levels associated with the screen.
 19. The one or more non-transitory computer-readable media of claim 15, wherein the execution of the computer-readable instructions cause the one or more processors to: analyze an image of the scene captured by the camera; and determine the first exposure level associated with the screen and the second exposure level associated with the portion of the scene based on the image of the scene captured by the camera.
 20. The one or more non-transitory computer-readable media of claim 15, wherein the one or more first target exposure levels comprises one of a first target exposure threshold or a first target exposure range, and wherein the one or more second target exposure levels comprises one of a second target exposure threshold or a second target exposure range. 